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ABSTRACT 


This  paper  defines  infinite  horlson  linear  programs  and 
presents  a procedure  that  vlU  approximate  the  optimal  solution 
of  almost  any  infinite  horizon  linear  program  that  has  a finite 
optimal  value.  In  addition.  It  Is  demonstrated  that  other 
procedures  for  calculating  optimal  solutions  will  not.  In 
general,  approximate  the  optimal  solution. 


/• 

APPROXIMATION  OF  OPTIMAL  SOLUTIONS  FOR  INFINITE  HORIZON  LINEAR  PROGRAMS 

by 

Richard  C.  Grlnold 


0.  INTRODUCTION 

This  paper  examines  long  range  planning  models  that  can  be  presented 
as  linear  programs  over  an  Infinite  planning  horizon.  The  main  results 
characterize  problems  that  have  finite  optimal  values  and  establish  procedures 
for  approximating  optimal  solutions  by  solving  a T period  linear  program. 
Three  possible  approaches  to  this  task  are  examined  and  It  Is  demonstrated 
that  only  one  procedure  leads  to  solvable  T period  problems.  The  T 
period  problem  Is  designed  by  decoupling  the  infinite  problem  Into  the  sum 
of  a T period  problem  and  another  Infinite  problem  that  commences  at  time 
T;  call  these  problems  1 and  2.  Any  feasible  solution  of  problem  1 produces 
an  input  into  problem  2.  The  scheme  calculates  an  approximate  salvage  Value 
of  the  Input  from  problem  1 to  problem  2.  This  salvage  value  is  then  included 
in  the  objective  of  problem  1.  As  T Increases  the  error  In  calculating  the 
salvage  value  becomes  less  and  less  significant.  The  main  conclusion  of  the 
paper  are  easily  stated:  (1)  the  only  defensible  way  to  solve  Infinite  horizon 

linear  programs  is  by  the  approximation  technique  outlined  in  section  four; 

(2)  duality  Is  not  the  primary  consideration  in  the  study  of  Infinite  horizon 
linear  programs;  (3)  the  calculation  of  an  equilibrium  optimal  policy*  If  one 
In  fact  exists,  does  not.  In  general,  assist  in  the  solution  of  a discounted 
criterion  problem. 

The  remainder  of  this  section  Introduces  the  problem,  summarizes  results, 
relates  this  work  to  others,  and  describes  the  notatlonal  conventions  used 


In  the  paper. 
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Tine  Is  discrete  t ■ 0,1,2,  . At  any  time  t the  state  of  the 

system  Is  s . Decisions  are  constrained  by  the  relations  Ax^  - s , 

> 0 . If  decision  x^  la  taken,  then  a reward  with  time  zero  value 
aSx^  is  received  and  the  state  at  time  t -f  1 Is  b + Kx^  . A and  K 
are  m x n matrices,  p an  n vector,  b an  m vector,  s an  m vector 
and  a a positive  scalar  less  than  1. 

If  the  Initial  (time  zero)  state  is  s then  the  Infinite  sequence  of 
decisions  (x^)  Is  constrained  by 

Ax  - s , X >0 
o o ■ 

(1) 

Ax^  ■ b + Kx^  - ,x^>0  t>l 
t t-1  t ■ ■ 

Let  X(s)  be  the  set  of  (x^)  that  satisfy  (1).  For  any  {x^>  define 

T . 

(2)  p((x.})  ■ lin  inf  I a px  . 

T -►  » 0 ^ 

The  optimization  problem  under  consideration  Is 

Maximize  p({x^}) 

(3) 

subject  to  {x^}  e X(s)  . 

Let  e be  a vector  of  ones,  a summation  vector.  A sequence  {x^} 

Is  called  a-oonvergent  If  the  series  of  increasing  nonnegative  terms 

J a ex^  converges  to  a finite  limit. 

Denote  X(s)  as  the  set  of  a-convergent  {x^}  e X(s)  , and  note  that 

OQ 

p({x  })  - I O^x  for  {x  } e X(s)  . 

0 
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Section  one  presents  assumptions  made  In  the  paper  and  comments  on  their 
Immediate  consequences  and  the  problem  of  verification.  These  assumptions 
hold  throughout  and  are  not  restated  for  each  result.  Section  two  defines 
the  optimal  value  of  problem  (3)  as  a function  of  s . The  consequences  of 
the  assumptions  not  holding  are  investigated  In  sections  two  and  three. 

Section  three  contains  an  important  result:  If  a program  {x^}  is  feasible 

but  not  a-convergent,  then  {x^}  is  a bad  program  in  the  sense  that 

p({Xj.})  - - » . 

Section  four  describes  a sequence  of  solvable  T period  linear  programs 
that  can  be  used  to  solve  (3).  The  optimal  solutions  and  optimal  values  of 
the  T period  problems  converge  to  an  optimal  solution  and  the  optimal  value 
of  the  Infinite  problem.  Section  five  presents  sufficient  optimality  crlterea 
and  comments  on  duality  and  the  establishment  of  necessary  conditions  for 
optimality.  Section  six  examines  the  question  of  calculating  optimal  equilibrium 
policies;  l.e.,  x^  « x for  all  t . The  appendix  contains  the  statement  and 
proof  of  two  lemmas  that  are  used  In  the  main  body  of  the  study. 

This  paper  Is  based  on  r-revlous  work  by  Manne  [13],  Hopkins  [11], [12], 
Hopkins  and  Grlnold  [18],  and  Evers  [3].  The  work  of  Evers,  [3]  motivated 
this  study  and  is  the  genesis  for  the  important  assumption  II  In  section  1. 
Assumptions  like  II  are  Implicitly  made  in  [13]  and  [8],  however,  Evers  was 
the  first  to  state  and  exploit  fully  this  type  of  assumption.  The  assumptions 
as  stated  In  section  1 are  satisfied  by  the  class  of  problems  considered  In  [3]. 
in  fact,  in  sections  2 and  3 It  Is  demonstrated  that  the  assumptions  are  nearly 
the  most  general  possible.  Except  for  certain  boundary  cases,  failure  of 
assumption  ] or  II  Implies  the  optimal  value  of  (3)  Is  either  + “ or  - » . 

Hie  theorem  in  section  2 generalizes  a result,  theorem  5.2,  [3].  Evers  [3], 
with  more  restrictive  assumptions.  Is  able  to  establish  several  attractive 
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results  that  characterize  optimal  solutions.  However,  since  the  main  purpose 
of  this  paper  Is  to  delineate  the  class  of  solvable  problems  and  to  construct 
a viable  solution  procedure;  the  more  restrictive  assumptions  of  [3]  are 
not  necessary  to  accomplish  this  objective. 

The  approximation  procedure  used  In  section  four  can  be  found  In  [2],  and 
In  (6]-[8],  and  [10]-[13].  In  [8]  and  (2],  ti  j procedure  solves  the  infinite 
problem  exactly  when  T - 0 . There  are,  however,  no  general  converge^nce 
results  available  prior  to  the  theorem  In  section  tour. 

Work  on  optimality  conditions  can  be  found  In  [3] , [5] , [6] , [11]-[13] , [ 15] . 
The  sufficiency  result  Is  Identical  In  spirit  to  [12],  and  the  remarks  on 
duality  are  consistent  with  [9].  In  section  six  optimal  equilibrium  policies, 
x^  - X for  all  t , are  discussed.  This  question  has  been  studied  In 
[1],[41  and  [101. 

The  following  notatlonal  conventions  are  used  in  the  paper.  The  symbol 
> 0 means  nonnegative;  ^ semi-positive;  and  > strictly  positive.  The 
vector  e Is  used  for  summmatlons;  each  element  of  e equals  one.  Script  S 
Is  used  as  a subsequence  of  Integers  and  ^ means  that  is  a 

refinement  of  5^  . Frequent  use  Is  made  of  the  Heine-Borel  theorem;  every 
sequence  in  a closed  bounded  set  of  R**  has  a limit  point  in  that  set. 
Equations  are  numbered  within  each  section. 


5 


1.  ASSUMPTIONS 

This  section  presents  and  explores  the  assumptions  used  In  all  the 
following  sectlona.  The  aasumptlona  are 

I:  There  exists  an  a-convergent  aolutlon  {x^}  e X(s)  . 

II:  For  every  X , 0 < X < a , there  exists  a solution  (u,v)  of  the 

equations 

(1)  u(A  - XK)  ■ V + p , V > 0 . 

Ill:  One  of  the  following  holds. 


(1) 

uA*v  + p ,v>0  has  a aolutlon 

(11) 

For  some  0 < X < a , (u,v) 

solve 

(1)  and  either 

uA  > p or  uA  > 0 . 

(111) 

For  some  0 < X < o , (u,v) 

solves 

(1)  and  uAx^  > 0 

or  (v  + XK)x^  > 0 for  all 

{x^}  c 

X(s)  . 

We  shall  see  in  section  3,  that  If  II  and  III  hold  and  I does  not  then 
either  the  problem  Is  Infeasible  or  each  feasible  solution  {x^}  has 
p({x^})  - - « . A similar,  not  quite  as  strong,  converse  statement  Is  true. 
Suppose  I Is  true  and  for  some  0 < X £ a , there  does  not  exist  a solution 
(u,v)  of 


u(A  - XK)  «p  + v,  v>0. 

In  this  case,  as  Is  demonstrated  In  section  2,  the  problem  Is  unbounded  for 
almost  any  value  of  s . 

Both  I and  II  cannot  be  verified  In  general.  However,  It  Is  possible  to 
gain  some  Information  about  the  assumptions.  For  example.  If  I Is  satisfied, 
then  there  exists  a solution  x of  the  system 


(2) 


(A  - oK)x  ■ s + T — 
1-a 


f 


X > 0 
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To  see  this,  multiply  the  t^  constraint  of  (1:1)  by  , sum  and  note  that 
00 

X " ^ a^x  satisfies  (2).  If  (2)  Is  not  feasible,  then  I falls.  There  Is 
0 

no  conclusive  test  to  verify  I.  As  In  linear  programming  this  Is  equivalent 
to  another  Infinite  horizon  linear  program. 

Assumption  II  Is  difficult  to  verify.  However,  II  and  III  (11)  will  be 
satisfied  If  there  exists  a solution  of  either 

(3)  u(A  - oK)  -v-p,v>0,uA>p 

or 


(4)  u(A  - oK)  -v»p,v>0,uA>0. 

For  (3)  note  that 

A - AK  - ^1  - ^ 

Thus  for  all  0 < A < a , we  have 

u(A  - AK)  ^ ~ V + p . 


^ A + ^ (A  - uK)  . 


For  (4)  note  that 


u(A  - AK)  > u(A  - oK) 


V + p . 


General  verification  of  II  requires  that  there  does  not  exist  a solution 
of  the  generalized  eigenvalue  problem 

with  x,y  ^ 0 , and  0 < A ^ a . 


Aasuoptlon  III  is  a patchwork  of  special  cases  to  cover  the  apparent 
loophole  in  II  (X  ■ 0) . Note  that  III  (i)  will  be  satisfied  if  II  holds 
for  all  X , 0 < X < a , or,  more  directly,  if  either  Ax  0 , x ^ 0 
has  no  solution  or  Ax  - 0 , x >_  0 Implies  px<0,or  Ax-0,x^0, 
px  > 0 implies  Kx  " 0 . Assumption  III  is  only  necessary  if  Ax  ■ 0 , 
x^O,px>0,Kx)(0  has  a solution.  Items  (11)  and  (ill)  are  directed 
to  this  particular  case.  It  is  a reasonable  conjecture  that  assumption  III 
is  not  necessary.  At  present,  theorem  1,  section  3,  cannot  be  proved  without 
III,  and  it  has  not  been  possible  to  construct  a counter  example  based  on 
the  failure  of  III.  The  details  in  III  (li)  and  (ill)  are  Included  to  cover 
all  the  cases  used  in  Evers  [3],  and,  more  significantly,  to  indicate  the 
failure  of  a possible  solution  procedure:  see  example  3 in  section  4.  In 
particular,  III  (ill)  covers  the  "primal  directed"  case  [3]  in  which  A ■ (B,I), 
K ■ (H,0)  and  p ■ (q,0)  . Each  row  of  B is  either  nonnegative,  or  the 
corresponding  row  of  H and  element  of  b are  nonnegative.  This  assures 
Ax^  i 0 t and  {x^}  c X(s)  . Moreover,  u(A  - oK)  - v + p 

Implies  u > 0 ; therefore  III  (ill)  is  satisfied. 
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2.  THE  OPTIHAL  VALUE  FUNCTION 

Define  V(s)  to  be  the  optimal  value  of  the  optimization  problem  aa  a 
function  of  the  Initial  state  a . 

* 

(1)  V(s)  - 8Up{p({x^})  I {x^}  c X(s)} 

Let  F ■ {s  I X(s)  , be  the  set  Initially  feasible  states.  If  s ^ F , then 

V(s)  Is  defined  to  be  - <•>  . This  section  demonstrates  that  V Is  convex  and 
satisfies  a dynamic  programming  functional  equation.  The  section  closes  by 
examining  the  consequence  of  assumption  II  not  holding. 

Let  dom  V ■ {s  | V(s)  >-<»}.  Both  F and  dom  V are  convex;  and 
dom  V C F . Let  F C F be  the  set  of  s such  that  I holds.  F C dom  V C F . 

Proposition  1; 

V(s)  Is  concave. 

Proof; 

Following  Rockafeilar,  [14],  p.  25,  note  that  V Is  concave  If  and  only 
12  12 

If  all  X,Y  ,Y  fS  satisfying 

0 < X < 1 , V(s^)  > Y^  , V(s^)  > Y^ 


Imply  that 


V((l  - X)s^  -r  Xs^)  > (1  - X)y^  + Xy^  . 

Select  {x^}  e X(s^)  for  1 ■ 1,2,  such  that  y^  < p({*^))  < V(s^)  . Note 
that  {(1  - X)xJ  + Xx^}  I X((l  - X)s^  + Xs^)  and  that 

V((l  - X)s^  + Xs^)  > p{(l  - X)x^  + Xx^}  > 

(1  - X)p{xJ}  + Xp{x^}  > (1  - X)y^  + Xy^  . 
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In  addition,  V ■atlaflaa  a dynaaic  prograaBlng  functional  aquation. 
Propoaltlon  2; 

On  tha  convax  act  F , V aatlaflaa  tha  functional  equation. 

(2)  V(a)  - aup{px  + aV(b  Kx)} 

Ax  ■ a , X > 0 . 


Proof; 

If  8 e F , and  Ax  ■ a , x > 0 , then  V(a)  > px  -f  aV(b  + Kx)  . The 
contrary  conclusion  would  violate  (1).  However,  for  any  c > 0 , It  Is  possible 
to  find  (x^}  e X(s)  such  that 

V(a)  > p({x^})  - px^  + ap({x^^j^})  > V(s)  - c . 

Since  {x_.}  e X(b  + Kx  ) , aV(b  + Kx  ) > op({x_,})  . Therefore 

Ctl  O O ■ CTl 

V(s)  > px  + aV(b  + Kx  ) > V(s)  - c , 

■ O O ■ 

i.e.,  V(s)  solves  (2).  || 

This  next  proposition  exaadncs  the  inplications  of  a failure  In  assumption 
II.  Failure  of  II  Implies  for  sosm  0 < X < a , u(A  - XK)  ■ v p , v > 0 
has  no  solution.  To  rule  out  a border  line  case,  total  failure  here  will 
mean  u(A  • XK)  • v -f  p , v > 0 has  no  solution. 

Proposition  3; 

If  I holds  and,  fur  some  X , 0 < X < a , the  system 

u(A  - XK)  > p 

has  no  solution,  then  V(s)  - 4-  ■>  for  s in  the  relative  Interior  of  dom  V , 


10 


and  V(«)  - - • for  s ^ don  V . V can  only  ba  finite  on  the  relative 
boundary  of  don  V . 

Proof; 

If  u(A  -XK)  > p haa  no  solution,  then  Parkas  Lenma  Implies  (A  - XK)y  ■ 0, 
y ^ , py  ' 0 has  a solution. 

Suppose  s e F and  {x^}  e ]T(s)  . Then  {x^  -f  y/X^}  e X(s  Ry/X), 
and  p({x^  -f  y/X^})  ■ •f  • . Thla  Implies  V(s)  Is  an  Improper  concave 
function  and  according  to  Rockafellar,  [14],  Theorem  7.2,  V can  only  be 
finite  on  the  relative  boundary  of  Its  domain.  | | 

The  following  example  Illustrates  the  result. 

Example  1; 


V(s)  ■ - • If  s 0 
otherwise 
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If  I f«ll«,  then  theorm  1 of  the  next  section  Indicates  that  V(s)  ■ - • . 
Either  X(s)  ■ ^ • or  there  ere  no  a-convergent  solutions  In  X(s)  . 

Theorem  1 states  that  (x^}  not  a-convergnet  Implies  p({x^})  ■ - • . 
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3.  THE  CLASS  OF  OPTIMAL  SOLUTIONS 

The  section  demonstrates  that,  without  loss  of  generality,  problem  (1:3) 
is  equivalent  to 

(1)  Max  I 

0 ' 

subject  to  {x^}  c ?(s)  . 

Solutions  that  are  not  a-convergent  are  not  potentially  optimal  solutions. 
Tlieorem  li 

If  {x^}  c X(s)  Is  not  u-convergent,  then  p({x^})  ■ - . 

Proof: 

Let  X be  the  smallest  number  such  that  {x^}  is  y convergent  for 

all  0 < Y ^ ^ 1 Ifi  the  radiuB  of  aonvergenoe  of  the  power  series 

00 

\ y^ex  . The  proof  first  shows  that  assumption  II  implies  that  any  solution 
0 ^ 

with  radius  of  convergence  \ , 0 < X ^ o has  p({x^})  ■ - » . The  remainder 
of  the  proof,  and  assumption  III,  are  needed  to  deal  with  {x^}  that  have 
a radius  of  convergence  equal  to  zero. 

00 

Let  X be  the  radius  of  convergence  for  the  power  series  ^ Y^ex  . 

0 ^ 

Assume  first  that  0<X^a.  If  X>a,  then  {x^>  is  u-convergent.  If 
the  t constraint  of  (1:1)  is  multiplied  by  X*^  and  the  first  T 
constraints  are  sunned  the  result  Is: 

T T 

(A  - XK)  I \^x.  - 8 + i xS  - 
0 1 


(2) 
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Let  M-  - ^ X^ex  , end  divide  (2)  by  tC.  • Consider  leone  1 In  the  eppendlx 

i g fc  1 

with  e^  ■ X^ex^  . Fron  leone  1,  there  exists  a sequence  S , such  that 
t t o 

T 

T r t 

•j/tij  ■ X -►  0 . Thus  on  a finer  subsequence  C ^ X x^/Hp  x ^ 0 

and 

(A  - XK)x  - 0 , X ^ 0 . 

From  II,  there  exist  (u,v)  such  that 

u(A  - XK)  “ p + V , V > 0 . 

It  follows  that 


px  + vx-0  ,vx>0,px<0. 


T 

For  T f S,  , — — -»  px  , thus  11m  inf  I X^px  » - * , From  leoma  2 in 

^ T -►  * 0 ^ 

the  appendix,  with  p > ^ , it  follows  that  p({x^})  * - » . 

QD 

Y ex  is 
0 ^ 

equal  to  zero.  For  any  X > 0 , (2)  will  have  a solution.  If  (2)  is  divider^ 
T ^ 

by  ^ " I X ex  , then  there  exist  a subsequence  5.  and  vectors  x(X)  , 

^ 0 ^ 

y(X)  such  that 


-►  x(X)  > 0 


T E S 


(3) 


y(X)  >0  T e S 
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(A  - AK)x(X)  - -XKy(X) 
ex(X)  - 1 , 
x(^)  i y(^)  • 

For  i ^ ^ " x(l/W  » ■ y(l/k)  • There  exists  a 

subsequence  S , and  points  x and  y such  that 

x*'->x^O  keS 

(4)  y*^  -►  y > 0 k e S 

Ax  > 0 . 


From  assumption  III  part  (1),  there  exists  (u,v)  such  that 

uA«v  + p,v>0. 


It  follows  that  vx  > 0 
px^  < 0 . For  X ■ 1/k  , 
such  that 


px  < 0 , and  for  k sufficiently  large 
T ^ 

\ X ex  , there  exists  a subsequence  S 
0 ^ 


It  follows  that  llffl  Inf  ^ X%x  > - • , and  from  lenna  2,  that 
T » 0 ' 

p({Xj.})  - - ® . 

It  remains  to  consider  III  (11)  and  III  (111) . From  (2)  either 
T T-1  T 

us  + ^ X^ub  ■ I X^vx  + [ X%x  + X^(u  + XuK)x_ 

1 0 0 


(5) 
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or 

r+1  T T 

(6)  us  + ^ X^ub  - I X^vx  + I X^px  + X^''’^uAx^.  . 

1 0 0 ' 

If  either  III  (11)  or  (111)  holds,  then  either  (v  -f  XuK)x^  > 0 or 
uAx^  > 0 . Thus  either 

us  + X ub  >_  I X^vx  + I X*^px 
1 0 0 

or 

T+1  T T 

ua  + I X ub  > I x'vx  + I X px  . 

1 0 0 

In  both  cases  the  left  hand  quantity  is  convergent.  If 

^ t ^ t 

not  a~convergent,  then  ^ X^vx  •♦■  + » , therefore  ^ X^px 

o'  o' 

lemma  2,  p({x^})  - - « . | | 

Corollary; 

If  I fails  then  V(s)  - - ® . 

Proof; 

If  X(s)  • 4i  , then  V(s)  ■ - ® . Othervise  every  {x^}  e X(s)  Is  not 
a-convergent,  by  theorem  1 p((x^})  ■ - ® , thus  V(s)  ■ - ® . [( 

From  (4)  In  the  proof  of  theorem  1 one  can  conclude  several  things. 
First,  if  Ax  " 0 , X 0 has  no  solution,  then  each  {x^}  e X(s)  Is 
X-convergent  for  some  X > 0 . Second,  If  Ax  - 0 , x ^ 0 implies  px  < 0 , 
then  III  (1)  is  satisfied,  and  the  arguments  following  (4)  apply. 


{x^}  is 
- ® , and  by 
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Third,  If  Ax>0,x^0,px>0  lapllca  Kx  - 0 , 
since  (A  - aK)x  ■0,x^0,px>0  has  a solution 
and  (ill)  are  only  needed  if  Ax"0,x^0,px>0 


then  II  does  not  hold 
The  cases  III  (11) 
and  Kx  0 , 


has  a solution. 
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I 

4 

4.  FINITE  HORIZON  APPROXIMAnOHS 

This  section  considsrs  s finite  (T  period)  horizon  approxlnetlon 
progrea  end  demons tretes  that  the  optimal  solutions  of  the  T period 
problems  converge  to  the  optimal  solution  of  the  Infinite  horizon  problem. 

The  T period  problem  Is 


Max  I aSxf 
0 ^ 


(1) 


T T 

subject  to  AXq  • s , Xq  > 0 


AxJ  - b + i 0 


(A  - ^ i i 0 


T 

Define  V (s)  as  the  optimal  value  of  (1). 
Proposition  4; 

Problem  (1)  has  an  optimal  solution  for  all  T , and 

(1)  v'^’Cs)  > v'^^^(s)  > V(s)  . 

(11)  v'*’(s)  ■ max[px  + aV^(b  Kx)J 

Ax  - s , X ^ 0 . 

m 

Proof; 


The  dual  of  (1)  Is 
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T-1 

Minimize  uje  + ] u^b  + u^b/l-o 


(2) 


subject  to  uJa  > a^p  + for  0 < t < T - 1 


uJ(A  - oK)  > a'*‘p 


T t 

Let,  by  II,  (u,v)  satisfy  u(A  - oK)  - v ■ p , v > 0 . Then  - a u 
Is  feasible  for  (2).  Also,  suppose  that  {x^}  c 3T(s)  , then 


Is  feasible  for  (1).  Since  (1)  and  (2)  have  feasible  solutions,  they  have 

optimal  solutions. 

T 

Suppose  u^  solves  the  T period  approximation,  then 


T+1 


u‘  t < T 


du‘  t-T+1' 


is  feasible  for  the  T 4-  1 period  dual,  vith  value 


“o®  I “t**  ■ “o*  ^ “t** 


T+1  T 

Therefore  V (s)  < V (s)  . 

For  any  {x^}  e X(s)  , a feasible  solution  of  (1),  using  (3),  can  be 

T 

constructed.  Thus  from  Theorem  1,  V (s)  > V(s)  . 

T 

Item  (11)  follows  directly  from  the  definition  of  V (s)  . || 
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Theorwi  2; 

T 

For  each  T , let  {x^}  be  the  optlnal  aolutlon  of  (1),  where 
■ 0,t  > T . 

(1)  There  exists  an  optimal  solution  {x^}  of  the  Infinite  horizon 
problem. 

(11)  For  every  t , there  exists  a sequence  such  that 
T 

x^  > x^  for  0<t<T,TfS 

t t « ■ » T 


(ill)  V^(s)  - I a^pxj  -►  V(s)  « I a%x  . 
0 0 


Proof; 


Let  ^ a ex^  and  note  that 

0 


(5) 


(A  - oK) 


s + 


gb 

1-g 


Suppose  Hj.  » + «• 
Sj  C S , and  a vector 


on  some  subsequence 
z such  that 


S 


then  there  exists  a subsequence 


(6) 


I - * T c S 

(A  - aK)z  - 0 , z ^ 0 . 


Since  II  is  valid,  there  exists  (u,v)  such  that  u(A  - oK)  ■ p + v , v > 0 . 

T 

Thus,  0 ■ pz  + vz  , vz  > 0 ; supz  < 0 . For  T c , V (s)  ■ Hppz^  > V(s)  > - ® 

However,  i-  °°  , pz^  ->  pz  ^ 0 . Thus  contradiction  Indicates  that 

M - lim  sup  M_  < + * . 

T 
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X C T 

For  «ach  t , «x^  < M/a  , > 0 . Thus  there  exists  s sequence  S , 

T 

end  vector  x such  thst  x x for  T c S . In  general  there  exist 
o o o o 

s sequence  C end  vector  x^  ouch  thst 


^t  ^ *t  ^ » 0 < t < T , 


It  follows  thst  {x  } e X(s)  and  J a^ex  < M . To  verify  this  last  claim 

' o' 

note  that  for  each  T and  t , 


I a'ex^  < M - I a'exj  < M 

o'  T+1  ' 


t T 


The  limit  over  T e S , yields  \ a ex  < M . Thus  {x  } e X(s)  ; and 

T ^ t • t 


I a'px  - V (s)  > V(s)  > I a'px 
0 0 


In  the  limit 


lim  I a'pxj  ■ E + J a'px  , e > 0 . 
T-»  0 ' 0 ' 


If  e - 0 , the  proof  is  completed.  Suppose  in  contrast  e > 0 . Then  for 
each  T 


t T 

I a p(x  - X ) > t > 0 . 

0 

Let  u,v  solve  u(A  - aK)  ■ p + v , v > 0 . From  (5) 


us  + 


aub  r t 
1-a  J 


vx. 


T 

t • 


(6) 
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Sine*  {x^}  e X(s)  , 


(7)  us  + ^ - I a^vx  - I oSx  . 

Subtract  (7)  from  (6)  to  obtain 

flo  m 

I a%(xj  - X ) - J o^v(x  - xj)  > £ > 0 
0 ^ ' 0 ^ ' 

or,  for  any  t . 

J CO  0» 

(8)  I a\(x  - x^)  + I a^vx  > e + I a^vx^  > e > 0 . 

0 ^ ^ T+1  ' T+1  ^ 

However,  the  left  hand  side  of  (8)  can  be  made  arbitrarily  small, 
choose  T large  enough  so  that 


0 < ^ a^'vx  f.  e/3 

“ T+1  ' 


* « * 
Then  there  exists  a T such  that  for  all  T c 5 , T > T 

T * - 


Corollary  2; 

For  s t F 


I I o^v(x  - xb  I £ e/3  . II 
0 


V(s)  ■ max[px  •¥  aV(b  -f  Kx)] 
Ax  » a , X > 0 . 


First 
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One  tempting  variant  on  the  approximation  scheme  described  here  Is  to 
terminate  the  T period  problem  by  requiring  that  - x^  for  t > T . 
The  example  below  shows  this  Is  not  generally  applicable  since  I and  II 
may  be  satisfied  yet  (A  - K)x  > b » x > 0 may  not  have  a solution. 

Example  2;  A - 1,  K - 1.6,  b«l,  p«l,  a«l, 

..  ,.t+l  _ - «» 

X.  ■ ^ ia  feasible  and  T o ■ 2.222  ...  , yet 

t 0.6  g t ^ 

(A  - K)x  * 1 , x > 0 has  no  solution. 

Another  possible  way  to  approximate  the  infinite  problem  is  to  simply 
truncate  the  decision  process  at  time  T by  solving 

T 

u ? t T 
Max  I a px 

0 ^ 

T T 

subject  to  AXq  ■ s , Xq  > 0 

T T T 

Ax|;  - b + Kx*  , , X*  > 0 
t t-1  t» 

1 < t < T . 

m m 


However,  our  next  example  shows  that  problems  of  this  sort  may  be 
unbounded  for  all  T . 

Example  3; 


. _ fl  0 1 0l  p - (1,1,0,0) 

‘-(J)  --(J) 

-[MSS] 


a - »5 
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Note  that  I,  and  II  are  aatlafied.  In  addition.  III  fill)  la  satlafled 
alnce  for  every  {x^>  Ax^  - b + Kx^.j  > 0 • In  addition,  u(A  - oK)  - v + p , 
V > 0 haa  a aolutlon  with  u > 0 . 

To  see  the  problem  la  unbounded,  note  that  any  positive  multiple  of  the 
vector  y^  “ (0, 1,0,1)  can  be  added  to  x^  . Thus  arbitrarily  large  profits 


are  possl tie  at  time  T 


I 


5.  OPTIMALITY  CONDITIONS:  DUALITY 

This  section  briefly  consider  optleality  conditions,  establishes 
sufficient  conditions  for  optim&llty  and  cooaients  on  necessary  conditions 
and  duality. 

Theorem  3; 

If  {x^}  e X(s)  and  there  exists  {u^}  auch  that 

(1)  1 1 I i ^ 

(li)  u^A  - - a^p  + “tf !**■  ♦ ''t  - ° 

(lii)  v^x^  - 0 

then  {x^}  is  optimal. 

Proof ; 

From  (li)  and  (ill) 


V 


Ax. 


-I 

1 


“t+l*^t 


eo 


OD 


I u (b  + Kx  ) - J u Kx 
0 ^ 1 ' 


u s 
o 


+ I “ I “Sx 
1 ' 0 


If  {x^}  is  another  feasible  solution  not  in  X(s)  , then  p({]f^})  - - « . 

«D 

If  {X  } c X(s)  • then  from  (1)  J ® ; thus 

0 


I a^x  - u s + ^ u b - I V X + ^ a^px  > I a^x  . | | 
0 0 0 
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It  is  possible  to  define  e duel  infinite  horizon  ptogra’n. 


nlninlze  lln  sup  u s ^ u b 

T -*  « ° 1 


(1) 


subject  to 


V- 


o^p  + u 


t+1 


Proposition  A; 

The  mexlmua  primal  Infinite  horizon  problem  has  a value  greater  than  or 
equal  to  the  optimal  value  of  the  minimization  problem. 

Proof ; 

T 

Consider  the  dual  linear  programs,  (4:2).  The  optimal  solution  {u^} 

X t"*T  T 

determines  a feasible  solution  of  (1)  with  u^  ■ a Up  for  t ^ T . That 

T 

solution  has  value  V (s)  . Thus  the  Infimum  in  (1)  is  less  than  or  equal 
to  V(s)  . II 

Suppose  for  P > 1 , vi(A  - pK)  >0  , ub  < 0 has  a solution.  Then  if 
u(A  - aK)  > p , {a^u  + p^  solves  (1),  and  11m  sup(u  -f  u)s  + 

T -►  00 

T 

(a^u  + p^b  - - • . Thus  to  Insure  (1)  has  a finite  lower  bound,  one 

0 

must  at  least  require  (A  - pK)x  * b , x > 0 has  a solution  for  all  p > 1 . 

This  is  too  stringent  on  assumption. 

If  one  requires  that  dual  solutions  satisfy  1 1 1 1 < aSf  for  some 

M , then  the  infimum  of  (1)  is  equal  to  the  maximum  of  the  original  problem. 

T 

Recall  the  definition  of  u^  in  section  4 and  the  construction  of  the 

^ X 

optimal  solution  (x  } . Let  M_  * sup  1 1 u 1 1 . If  a subsequence  S 

If  ^ 

exists  such  that  < M for  T e S , then  it  is  possible  to  establish  the 
existence  of  (u^)  , such  that 
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(i)  a*^  I I u J I < M 

(il)  u^A  - - o%  + “t+1*^  • ® 

(111)  Vj.x^  - 0 

T 

(Iv)  There  exists  S such  that  ■*  for  TeS  ,0<t<T. 

T tt 

This  fact  is  easy  to  establish.  However*  It  Is  of  little  use  unless  conditions 
which  bound  can  be  deduced  from  the  data  A*K,p,b*a  . 


’i 


wrnmwmm$^ 
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6.  EQUILIBRIUM  SOLUTIONS 

In  [1],  [4]  and  [10],  the  problea  of  finding  an  equlllbrlua  optimal 
aolutlon  ■ X for  all  t waa  conaldered  and  aolvad  In  a moat  elegant 
way.  This  section  points  out  that  knovledge  of  an  equlllbrlun  optimal 
aolutlon  Is  not  particularly  helpful  in  solving  problem  (1:3). 

Suppose  the  system 


(1) 


(A  - K)x  - b , X > 0 


baa  a feasible  solution.  Then,  If  x , satisfies  (1),  {x^  ■ x)  e X(b  Kx) 

and  V(b  + Kx)  > . Now  consider,  tl«e  optimization  problea 


(2) 


Max  px 

subject  to  (A  - K)x  • b , x > 0 . 


Suppose  first  that  (2)  has  unbounded  solution;  i.e.,  py  > 0 , (A  - K)y  • 0 , 
y > 0 has  a solution.  Then  for  any  s e F 

V(a  + vKy)  > V(s)  + py  , 

since  {x  vy)  e X(s  + uKy)  for  all  (x  } e X(s)  . In  this  case  the  function 

^ ' * 

A A nv 

V Is  unbounded.  If  (2)  has  an  optimal  solution,  x , then  V(b  Kx  ) > . 

Now  suppose  an  equilibrium  optimal  solution  x exists;  I.e., 

(A  - K)x  ■ b , X > 0 , V(b  + KjO  “ . It  follows  that  either  x solves 

■ i-a 

(2)  or 

(3)  V(b  + Kx*)  > • V(b  + K30 

- 1-a  1-a 

* * 

Thus  the  suboptimal  solution  s ■ b 4-  Kx  , and  x^  ■ x for  all  t Is 
preferable  to  the  optimal  solution  s«b4>Kx,x  "?  for  all  t . 

e t 
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More  general  equilibrium  policies  can  be  discovered  by  allowing  a T 
period  cycle;  l.e.,  u - b -f  Kx^  . Solve  with  s as  a variable 

T 

Max  I a^px 
0 ^ 

subject  to  -Is  -f  Ax  ■ 0 • x >0 

o 0 ■ 

Ax^  - b + Kx^  ,,x^>0  l<t-^T 
t t-1  t ■ ■ ■ 

Is  ■ • b . 

T 

If  U is  the  optimal  value  of  (4),  and  s,x  , x_  is  the  optimal 

o T 

solution,  then 

1-a 

* * 

Since  b 4-  Kx  , x^  * x is  feasible  for  (1) . 

Thus  if  optimal  equilibrium  solutions  can  be  calculated  then  "better" 
suboptimal  solutions  can  be  calculated  using  (2)  and  (4).  These  calculations, 
however,  are  not  directly  useful  in  the  solution  of  the  original  optimization 
problem.  The  more  direct  procedure  outlined  in  section  4 yields  approximately 
optimal  value  and  an  approximately  optimal  first  period  decision  that  is 
consistent  with  the  initial  state  s . 
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7.  APPENDIX 

This  appendix  contains  two  leomaa  used  In  the  proof  of  theorea  1. 

Leana  1; 

If 


(1) 

“t  i * 

(11) 

00 

I A^a 

0 

T 

(ill) 

11m  1 

then  11a  Inf  - 0 . 


X ->  <*> 


r\  • 


Proof ; 

* 

If  the  result  Is  false,  there  exists  an  1 > e > 0 , and  T such  that 
for  all  T > T* 

m 


(1) 


Let  b 

o 


y and 
0 ‘ ‘ 


Therefore 


(2) 


T 

b-  ^ L y b^  for  all  T . 
T-  i t 


or 


(3) 
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It  can  be  established  by  Induction  that 

(4)  b^  > y(l  + for  T > 1 . 

(3)  denonstrates  that  (4)  Is  trua  for  T • 1 . To  check  T I note 
that 

•’m  i ^ •’t  i "(i  * ^ /o 

■ y(l  + u)^b  . 

o 

This  Indicates  that 


where  A 


However 


* 

T +T 

I. 

T +1 


which  Implies  ^ A^a  diverges.  | | 
0 


Lenma  2; 


T 

If  11m  inf  ^ a 
T ^ ® 0 ^ 


® and  0 : 1 


then 


11m  inf  f P^a 
T 00  0 ' 


Proof  t 


* j 

Trivial  If  p ■ 1 . Lot  r_  ■ ^ p o , and  p ■ 1/p  < 1 . Than, 

* 0 ' 


(5) 


I a - (1  - p)  f y V ^ p^^^ 

0 ^ o'  ^ 


If  11a  Inf  r_  > - ••  , then  r_  auat  be  bounded  below,  l.e.,  r_ 

X ^ T T 

for  all  T . Thua  from  (5) 


l*t  • - U)  I p‘  + M - M 


which  contradlcta  the  hypotheaea.  || 


■ V 
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